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Summary 

The  UK’s  MoD  has  funded  a  programme  of  applied  research1  to  explore  the  benefits  to  be  gained  from  using 
networks  of  simulator,  or  Synthetic  Training  Environments  (STEs),  for  multi-role  air  mission  training  (i.e 
collective  training2).  Within  the  UK  the  use  of  networked  simulation  in  this  context  has  become  known  as 
Mission  Training  through  Distributed  Simulation  (MTDS).  Within  the  US  it  is  known  as  Distributed  Mission 
training  (DMT).  The  Defence  and  Science  Technology  Laboratory  (Dstl)  and  QinetiQ  have  undertaken  the 
MoD  sponsored  research  via  a  series  of  trials.  The  trials  have  been  conducted  under  the  banner  heading  of 
RAPTORS3.  To  date  four  trials  have  taken  place;  Ebb  and  Flow,  SyCOE,  VirtEgo  and  SyCLONE.  All  have 
been  conducted  using  the  synthetic  Composite  Air  Operation  (COMAO)  test-bed  created  specifically  to 
assess  the  potential  of  MTDS.  Combat-ready,  front-line  aircrew  and  an  expert  White  Force  from  the  UK’s 
Air  Warfare  Centre  (AWC)  Tactical  Wing  and  Training  have  participated  in  all  four  trials.  The  research  has 
indicated  that  there  is,  potentially,  much  to  be  gained  from  the  use  of  networked  simulation  for  MTDS.  The 
question  remains  as  to  the  extent  that  participants  should  or  could  be  distributed  during  MTDS  exercises. 
This  is  particularly  pertinent  if  the  aspiration  is  to  use  networked  simulation  for  coalition  training,  because, 
of  necessity  this  would  require  some  training  participants  to  be  geographically  dispersed.  The  last  two  trials 
therefore  included  a  Wide  Area  Network  (WAN)  to  link  together  research  facilities  in  Canada,  the  UK  and 
US.  This  paper  will  discuss  the  outcome  of  these  trials  with  particular  reference  to  SyCLONE. 

Overview  of  VirtEgo  and  SyCLONE 

Trial  VirtEgo4  took  place  in  November  2001,  whilst  Trial  SyCLONE5  was  undertaken  in  January  2003.  The 
UK  elements  of  the  trials  were  funded  by  MoD  and  Strike  Command,  with  the  collaborative  aspects  carried 
out  under  the  auspices  of  The  Technical  Co-operation  Panel6  (TTCP).  As  in  the  previous  trials,  VirtEgo  and 


1  Under  the  sponsorship  of  the  Director  of  Equipment  Capability  (Theatre  Airspace) 

2  ‘Collective  mission  training’  is  defined  as  two  or  more  teams  training  to  interoperate  in  an  environment  defined  by  a  common  set 
of  collective  mission  training  objectives,  where  each  team  fulfils  a  different  military  role.  NATO  SAS-013  Study 

3  Research  into  Aircrew  Performance  and  Teamwork  using  Operationally  Realistic  Scenarios 

4  VirtEgo  -  Virt  stands  for  virtual  and  Ego  a  conscious  thinking  subject 

5  SyCLONE  -  Synthetic  CoaLition  Operation  in  a  Networked  Environment 

6  A  long-term  UK/US/CA  collaborative  project  is  being  developed  to  take  this  research  forward  under  the  auspices  of  two  groups  of 
the  Technical  Co-operation  Panel  (TTCP);  the  Eluman  Resources  and  Performance  Group,  Technical  Panel  2  (E1UM  TP-2,  Training 
Technology)  and  the  Aerospace  Systems  Group,  Technical  Panel  1  (AER  TP-1  Aerospace  Operational  Analysis  and  Simulation). 


Paper  presented  at  the  RTO  HFM  Symposium  on  "Advanced  Technologies  for  Military  Training", 
held  in  Genoa,  Italy,  13  -  15  October  2003,  and  published  in  RTO-MP-HFM-101. 
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SyCLONE  were  designed  to  execute  emulation  of  a  COMAO  and  incorporated  all  mission  phases:  tasking, 
brainstorming/planning,  briefing,  mission  execution  and  debriefing  [Ref  1],  The  trials  were  modelled  on  live 
collective  training  exercise  such  as  the  NATO  Tactical  Leadership  Programme  (TLP)  and  UK  Tactical 
Leadership  Training  (TLT)  and  the  US  Red  Flag  exercises.  The  major  caveat  being  that  the  live  collective 
training  exercises  have  participants  co-located.  For  elements  of  VirtEgo  and  SyCLONE,  participants  were 
geographically  dispersed.  This  was  a  critical  aspect  of  the  research. 

The  synthetic  environment  created  for  the  VirtEgo  and  SyCLONE  enabled  operational  aircrew,  based  in  the 
UK,  Canada,  and  US  to  perform  a  COMAO  based  training  exercise.  For  all  missions,  the  rules  of 
engagement  (ROE),  and  special  instructions  (SPINS)  were  based  on  those  of  the  theatre  in  question  and  the 
missions  were  designed  to  enable  the  aircrew  to  fly  in  accordance  with  coalition  tactics  and  doctrine. 

For  all  four  of  the  RAPTORS  trials,  a  team  from  the  AWC  Tactical  Wing  provided  an  expert  White  Force 
(WF).  They  provided  a  critical  element  of  the  exercise  management  function.  They  also  provided  expert 
assessment  of  collective  performance  throughout  each  mission  day.  This  mirrors  the  role  they  perform  in 
facilitating  the  UK’s  national  air  collective  training  exercises. 

The  manned  participants  in  the  trials  formed  part  of  a  small  package  of  aircraft  flying  a  coalition  mission 
within  a  shared  synthetic  battlespace.  Other  elements  of  the  package,  specifically  Suppression  of  Enemy  Air 
Defence  (SEAD)  assets,  were  represented  during  mission  execution  by  computer  generated  forces  (CGF). 
During  planning  the  SEAD  element  lead  was  represented  by  a  WF  role  player.  For  each  mission  day  one  of 
the  participating  aircrew  was  selected  to  be  the  Package  Commander  (PC)  responsible  for  co-ordinating  the 
package’s  efforts  in  order  to  achieve  commander’s  intent. 

The  package  formed  part  of  a  much  bigger  offensive  air  support  operation,  with  other  coalition  missions 
being  flown  against  hostile  ground  and  air  threats  within  the  same  timeframe.  The  added  complexity  of 
greater  numbers  of  friendly  and  hostile  forces  was  provided  largely  by  CGF,  with  the  addition  of  some 
manned  air  threats. 

VirtEgo 

Trial  VirtEgo  was  the  first  trial  to  combine  both  training  and  research  thrusts.  VirtEgo  was  designed  as  a 
‘proof  of  concept’  for  preparing  front-line  Qualified  Weapons  Instructor  (QWI)  students  for  COMAOs  in 
readiness  for  the  two  week  operational  phase  (Ops  Phase)  of  their  CQWI  training  programme.  An  encrypted 
trans-atlantic  WAN  to  Air  Force  Research  Laboratory  (AFRL)  in  Mesa,  Arizona,  US  was  also  implemented 
to  allow  a  coalition  package  to  fly  together  in  a  shared  synthetic  battlespace.  The  WAN  included  a  ‘stealth’ 
link  between  AFRL  and  Defence  Research  and  Development  (DRDC)  Laboratories,  Toronto,  Canada.  An 
initial  distributed  planning,  briefing  and  debriefing  was  also  provided.  This  capability  utilised  commercial- 
off-the  shelf  (COTS)  technology  and  took  the  form  of  a  video-teleconferencing  (VTC)  system  and 
interactive  whiteboards  i.e.  a  SmartBoard™. 

The  SmartBoard™  provides  a  touch  sensitive  screen,  of  relatively  large  physical  area,  which  is  convenient 
for  multiple  users.  The  Smartboard™  technology  allows  users  to  write  on  the  board  as  though  it  were  a  dry 
marker  board;  these  inputs  can  then  be  transmitted  via  Microsoft  NetMeeting  to  other  networked 
SmartBoard™  .  Electronic  files  created  at  one  location  can  also  be  shared,  thus  for  example,  a  PowerPoint 
presentation  can  be  viewed  simultaneously  at  different  locations. 

The  findings  from  VirtEgo  suggested  that  it  was  possible  to  combine  training  and  research  objectives  within 
one  trial,  but  that  the  technology  must  first  be  robust  and  proven.  [Ref  2],  A  perceived  utility  in  pre¬ 
deployment  training  for  ab-initio  crews  and  general  COMAO  refresher  training  was  also  found.  However, 
the  ability  of  the  STE  to  support  mission  training  for  dispersed  participants  was  not  proven.  There  were 
technical  difficulties  and  front-line  crews  did  not  fly  the  AFRL  simulators.  This  may  have  biased  the 
findings. 
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SyCLONE 

From  the  UK  perspective,  Trial  SyCLONE  was  designed  to  explore  the  impact  of  distributed  versus 
colocated  mission  training  on  the  UK  participants  (aircrew  and  WF).  In  previous  trials  the  UK  players  had 
been  colocated7  [Ref  3].  The  coalition  aspects  of  the  research  were  maintained  via  long-haul  transatlantic 
links  to  both  AFRL  in  the  US  and  Defence  Research  and  Defence  Canada,  Toronto  in  Canada.  The  manned 
elements  comprised: 

For  Trial  SyCLONE  the  manned  participants  were: 

•  1  x  UK  Ground  Attack  (GA)  4-ship  (Jaguars) 

•  lx  UK  Air- to  Air  (A- A)  4-ship  (Tornado  F3s) 

•  1  x  US  Swing  role  4-ship  (F-16Cs) 

•  lx  Canadian  GA  4-ship8  (CF-18s). 

The  training  research  focus  in  SyCLONE  was  on  the  effects  of  distribution  on  planning  and  co-ordination 
and  ultimately  on  execution  of  the  mission  itself.  While  simulation  is  deemed  to  be  essential  to  explore  the 
effectiveness  of  the  mission  plan,  it  was  not  the  focus  of  training  research  in  this  trial.  Aircrew  may  already 
have  some  experience  of  distributed  briefing/debriefing  in  military  exercises,  such  as  Cope  Thunder. 
Collaborative  planning  of  complex  missions  is  not  something  that  aircrew  normally  carried  out  in  a 
distributed  way  during  operational  training. 

The  importance  of  the  planning  phase  can  often  be  overlooked  in  simulation  exercises  when  the  emphasis 
tends  to  be  on  the  mission  and  the  time  spent  in  the  simulator.  However,  anecdotal  evidence  suggests  that 
up  to  80  to  90%  of  the  benefit  of  participation  in  large  scale,  complex  exercises,  whether  live  or  simulated, 
comes  from  involvement  in  the  planning  process.  The  stated  objective  of  planning  is  to  generate  a  product, 
namely  the  plan.  Really  one  of  the  most  important  benefits  of  planning  is  the  provision  of  a  rigorous, 
structured  way  to  learn  about  a  problem  space  and  therefore  to  develop  judgement  [Ref  4], 

During  trial  SyCLONE,  UK  crews  planned,  briefed,  debriefed  their  missions  in  separate  locations  but  at  the 
same  site.  As  in  VirtEgo  they  were  using  technology  specifically  provided  for  the  purpose,  namely  VTC  and 
SmartBoards™  to  communicate  with  each  other.  The  WF  was  also  kept  separate  from  the  aircrews.  This 
meant  that  in  effect  the  distributed  STE  comprised  five  dispersed  nodes:  three  at  Bedford  UK,  one  at  AFRL 
US  and  one  at  DRDC  Canada.  A  schematic  is  given  in  Figure  1. 


7  Little  previous  research  has  been  done  on  the  impact  of  distribution  on  teamwork  and  team  effectiveness  in  real  world  task 
environments.  A  comprehensive  review  of  teamwork  research,  estimated  that  of  more  than  4,000  team  studies,  95%  were  to  do  with 
colocated  teams  performing  tasks  in  laboratory  conditions.  From  the  remainder,  very  little  can  be  deduced  about  the  consequences  of 
distribution  for  effective  real  world  training. 

8  The  CF-18  4-ship  comprised  two  man-in-the  loop  players  and  2  CGF  wing-men. 
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ORGANIZATION 


Figure  1:  Experimental  set-up  with  UK  forces  distributed 


SyCLONE  -  findings 

Analysis  was  conducted  using  a  variety  of  assessment  methods  and  tools.  These  included  questionnaires, 
interviews  and  extensive  use  of  subject  matter  expert  opinion  (the  WF).  A  preliminary  analysis  of  the  data 
indicted  that  SyCLONE  did  not  provide  as  fully  optimised  a  training  environment  for  the  UK  participants  as 
did  the  previous  three  trials.  There  were  a  number  of  variables  and  confounding  factors  that  contributed  to 
this  outcome.  These  included: 

•  Late  mission  days  for  UK  crews 

•  Late  tasking  of  aircrew  to  participate  in  the  trial 

•  Current  ops  tempo  placed  an  additional  burden  on  WF 

•  Use  of  new  technology,  particularly  for  distributed  planning,  briefing  and  debriefing 

•  Some  of  the  more  specific  findings  and  analysis  are  now  discussed  in  turn. 


Distributed  planning,  briefing  and  debriefing:  The  research  team  developed  a  very  comprehensive 
concept  of  operation  (conops)  for  distributed  Planning,  Briefing  and  Debriefing  activities  (PBD).  The  results 
of  questionnaires  given  to  aircrew  indicated  that  they  were  happy  with  the  instructions  given  on  how  to  use 
the  technology  but  did  not  feel  the  PBD  system  was  user-friendly  or  robust. 

The  results  also  indicated  that  dispersion  had  impacted  upon  training  value  with  the  greatest  effect  being  felt 
during  the  planning  phase,  resulting  in  a  more  ‘simplistic’  plan  being  produced.  This  is  not  necessarily  a 
desirable  outcome  and  certainly  does  not  exploit  some  of  the  more  obvious  benefits  of  a  complex  MTDS 
environment.  On  the  distributed  days,  problems  were  also  experienced  by  the  WF  who  felt  unable  to 
function  as  effectively  when  dispersed  from  crews.  They  believed  that  their  ability  to  monitor  and  where 
appropriate  guide  the  planning  process  was  compromised. 
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The  overall  opinion  by  UK  participants  was  that  whilst  the  PBD  technology  was  usable  it  detracted  from  the 
training  benefits.  The  preference  as  in  previous  trials  was  for  colocated  synthetic  COMAO  training, 
irrespective  of  whether  or  not  it  is  representative  of  real  world  operations. 

To  summarise: 

•  Distributed  planning  resulted  in  a  simple  and  less  integrated  plan  which  did  not  fully 
exploit  the  capabilities  of  the  other  package  elements 

•  The  WF  felt  the  Package  Commander  (PC)  was  not  able  to  communicate  effectively 
with  other  members  of  package 

•  The  WF  felt  the  technology  too  cumbersome  for  planning  process 

•  Aircrew  observed  not  as  fully  immersed  in  ‘total  experience’  as  in  previous  trials. 
No  real  ‘buy  in’  to  the  synthetic  experience,  difficult  to  maintain  suspension  of 
disbelief  through  all  phases,  hence  more  critical  of  the  simulation  than  in  previous 
trials 

•  Low  WF  Situational  Awareness  (SA)  on  aircrew  planning  process,  resulted  in  an 
inability  to  fully  optimise  the  training  environment. 

As  discussed,  the  UK  crews  did  manage  to  have  one  day  colocated  where  they  were  able  to  plan  together. 

The  WF  provided  an  assessment  of  the  aircrews  performance  during  planning,  briefing  and  debriefing  for 
each  of  the  four  mission  days.  A  summary  of  this  is  given  below  in  Table  1  and  clearly  indicates  that  the  day 
on  which  the  crews  were  colocated  (mission  day  four)  produced  a  more  comprehensive,  well  thought  out 
and  tactically  considered  plan. 
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Table  1:  White  Force  assessment  of  Trial  SyCLONE  Planning  Phases  over  Four  Mission  Days 


Mission  1  (distributed) 

Mission  2  (distributed) 

Mission  3  (distributed  but 

colocated  for  debrief) 

Mission  4  (colocated) 

Planning 

•  Not  able  to  assess  planning 
phase  adequately 

•  PC  not  able  to  communicate 
effectively  with  other  package 
members 

•  1st  day  was  perceived  as  battle 
with  unfamiliar  technology 

•  Not  able  to  assess  planning  process 
adequately 

•  Inadequate  technology  to  support 
overview  of  distributed  planning 

•  PC  couldn’t  communicate 
effectively  with  other  package 
members 

•  Technology  seemed  to  impact  on 
ability  of  aircrew  to  conduct  workup 
&  produce  sophisticated  plans. 

Much  frustration  with  the 
technology 

•  Result  is  a  simpler  plan  -  Force 

Flow  type  approach  -  not  able  to 
develop  an  integrated  detailed  plan 

•  Not  able  to  assess  planning  phase 
adequately 

•  Same  problems  as  previous  days  - 
very  little  SA  on  aircrew  planning 
process  but  slight  improvement  - 
more  bandwidth  &  better  audio 

•  Able  to  assess  planning  for 

UK  crews.  UK  planning 
resulted  in  better  plan  than  on 
previous  days  -  more  detailed 
in  plan,  more  tactical  thought, 
using  work  rounds  to 
overcome  tech  difficulties  to 
distributed  US  &  CA  nodes. 

•  Use  of  VTC  &  SmartBoards  " 
affected  distributed  planning 
process  -  technology  too 
cumbersome  to  use  in  normal 
planning  process 
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It  is  well  understood  that  a  good  plan  makes  for  a  better  mission.  The  WF  view  was  borne  out  by  subsequent,  in- 
depth  post-trial  analysis  of  a  number  of  key  events  that  occurred  during  SyCLONE’s  final  mission  day9. 
Aircrew  comments  also  indicated  that  this  day  was  preferred  to  the  previous  three  mission  days. 

These  observations  support  the  findings  from  previous  trials.  Previous  experience  of  colocated  trials  shows  that 
rapport  begins  to  develop  between  colocated  crews  after  two  days  together;  this  is  accompanied  by  a  measurable 
improvement  in  performance.  The  effects  are  illustrated  in  Figure  2;  this  figure  shows  some  of  the  results  from 
an  earlier  RAPTORS  trial  (trial  SyCOE10)  which  took  place  in  January  2001  [Ref  5].  The  data  shown  are 
derived  from  an  Assessment  Criteria  questionnaire,  which  asks  for  assessments  on  thirty-one  criteria  of  effective 
mission  performance.  These  criteria  are  shown  in  table  2. 


Figure  2:  SyCOE  aircrew  assessment  -  mission  1  vs  mission  2 

Aircrew  performance  started  from  a  good  baseline  in  mission  1  and  improved  significantly  in  mission  2.  This 
improvement  was  particularly  marked  for  three  areas: 

•  appropriate  review  of  tactics  as  a  result  of  lessons  learnt  in  previous  mission, 


9 


This  analysis  is  being  undertaken  by  QinetiQ  as  part  of  an  MoD  funded  study  entitled  Quantifying  the  Effectiveness  of  Collective 


Training. 


10  The  training  design  of  trial  SyCOE  provided  a  more  appropriate  comparison  to  SyCLONE  than  VirtEGO 
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•  how  well  aircrew  understood  and  implemented  the  briefed  operational  procedures 

•  ‘between  formation’  awareness  of  other  team’s  capabilities 


Figure  3  shows  results  from  trial  SyCLONE. 


Figure  3:  SyCLONE  aircrew  assessment  -  mission  1  vs  mission  2 

Aircrew  performance  during  mission  1  was  not  given  a  particularly  high  rating,  as  compared  to  mission  1  in 
SyCOE.  No  obvious  improvements  in  aircrew  performance  can  be  observed  in  SyCLONE  mission  2.  In  fact 
there  appears  to  be  a  decrement  in  performance  in  some  areas,  particularly: 

•  appropriate  review  of  tactics  as  a  result  of  lessons  learnt, 

•  effectiveness  of  the  plan, 

•  how  well  each  role  appeared  to  understand  where  they  fitted  into  the  'bigger  picture'  of  the  COMAO 
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Table  2: 

Mission  effectiveness  -  performance  assessment  criteria 

Performance  Assessment  Criteria 

01 

How  effective  was  the  plan? 

02 

How  comprehensive  was  the  brief  ? 

03 

To  what  extent  were  all  eventualities  explored  and  addressed  during  the  planning  phase? 

04 

How  effective  were  the  tactics  employed  during  the  mission? 

05 

Were  relevant  lessons  learnt  and  actions  thoroughly  debriefed? 

06 

How  well  was  the  expertise  available  during  the  planning  phase,  utilised  by  the  PC? 

07 

How  well  were  the  needs,  workload  and  time  constraints  of  the  other  roles  taken  into  account  by 
the  PC  during  planning? 

08 

How  well  did  each  role  appear  to  understand  where  they  fitted  into  the  'bigger  picture'  of  the 
COMAO? 

09 

How  effective  were  the  responses  to  injects  and  self-generated  problems  during  the  mission? 

10 

How  aware  were  the  aircrew  of  relevant  events  and  problems  that  could  impinge  on  the  mission 

11 

Given  the  knowledge  available  to  them  at  the  time  how  well  did  aircrew  response  to  events  & 
problems  that  could  impinge  upon  the  mission? 

12 

How  well  was  formation  integrity  maintained  throughout  the  mission? 

13 

How  good  was  the  comms  discipline  within  the  formations? 

14 

How  effective  were  the  formation  leads  in  co-ordinating  the  assets  within  their  formations? 

15 

How  good  was  the  comms  discipline  between  formations? 

16 

How  effective  was  the  PC  in  co-ordinating  the  assets  within  the  package? 

17 

How  well  were  correct  tactics  employed  against  SAM  or  A-A  threats? 

18 

How  appropriate  were  any  review  of  tactics  made  as  a  result  of  lessons  learnt? 

19 

How  well  were  the  role-specific  tasks  demonstrated? 

20 

How  well  did  the  aircrew  understand  and  implement  the  briefed  operational  procedures? 

21 

To  what  extent  was  the  overall  effectiveness  of  the  group  positively  influenced  by  the  PC 
leadership? 

22 

To  what  extent  did  the  PC  appear  to  benefit  from  the  experience  of  command? 

23 

How  appropriately  did  the  others  respond  to  the  PC  leadership  style? 

24 

To  what  extent  were  the  appropriate  responses  made  to  in-flight  injects  and  self-generated 
problems? 

25 

To  what  extent  was  appropriate  flexibility  demonstrated? 

26 

How  much  confidence  to  individuals  appear  to  have  in  their  own  capabilities? 

27 

Within  formations,  how  much  awareness  did  element  leaders  and  their  no.  2s  appear  to  have  of 
each  others  capabilities? 

28 

To  what  extent  did  formation  members  appear  to  be  confident  enough  to  rely  on  each  others 
actions? 

29 

Between  formations,  how  much  awareness  did  teams  appear  to  have  of  each  others  capabilities? 

30 

To  what  extent  did  the  different  formations  appear  to  be  confident  enough  to  rely  on  each  others 
actions? 

31 

To  what  extent  were  the  overall  objectives  of  the  mission  achieved? 
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Trust  and  confidence:  Collective  training  exercises,  whether  live  or  synthetic,  should  endeavour  to  support  the 
development  of  inter-role  trust  and  confidence.  This  is  an  essential  component  of  interoperability  in  theatre.  To 
quote: 


‘The  secret  of  a  successful  air  campaign  is  interoperability 
and  the  most  important  component  of  interoperability  is  trust’ 

Air  Cdre  Stu  Peach,  Commandant  Air  Warfare  Centre 

An  apparent  inability  of  UK  and  US  crews  to  develop  trust  via  good  inter-team  cohesion  and  build  rapport  was 
observed  during  SyCLONE.  A  questionnaire  given  to  UK  aircrews  asked  them  to  provide  trust  and  confidence 
ratings  for  the  other  roles  that  were  participating  in  the  trial.  It  also  asked  them  to  rate  the  level  of  confidence 
they  had  in  the  ability  of  these  other  players  to  contribute  to  mission  success.  Pre  and  post  trial  ratings  of  trust 
and  confidence,  made  by  UK  aircrew,  were  analysed  and  are  presented  in  Figures  4  and  5.  Figure  4  shows  pre 
and  post  trial  ratings  for  trust  while  Figure  5  shows  pre  and  post  trial  ratings  for  confidence. 
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Pre  vs  Post  Trial  SyCLONE  -  UK  Crews  Trust  Ratings 


□  Pre  □  Post 


Level  of  trust  in  F3 


level  of  trust  in  Jags  level  of  trust  in  FI 6s 

Figure  4:  Pre  v  Post  trial  UK  aircrew  ratings  for  trust 
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Pre  vs  Post  Trial  SyCLONE  -  UK  Crews  Confidence  Ratings 


□  Pre  □  Post 


confidence  levels  in  F3  confidence  levels  in  Jags  confidence  levels  in  FI  6s 


Figure  5 :  Pre  v  Post  trial  UK  aircrew  ratings  for  confidence 

The  results  indicted  that  whilst  trust  and  confidence  had  increased  for  the  UK  participants,  it  had  reduced  for  the 
FI  6  role  (the  element  that  was  dispersed  for  the  complete  duration  of  the  trial).  This  effect  deserves  careful 
consideration. 

Trust  and  confidence  between  the  UK  crews  may  have  been  able  to  develop  through  social  interaction,  even 
though  attempts  were  made  to  keep  the  A-A  and  GA  formations  apart  as  much  as  possible.  Thus  aircrew  were 
sharing  the  same  hotel  and  took  meals  together  during  the  trial  day.  It  is  also  worth  bearing  in  mind  that  the  UK 
aircrew  did  manage  to  have  one  day  colocated  where  they  were  able  to  plan  together. 

The  reputation  of  the  capability  of  F-16  as  weapons  platforms  may  have  impacted  upon  pre-trial  ratings. 
However  the  lack  of  social  interaction  and  some  of  the  problems  encountered  with  the  PBD  technology  meant 
that  trust  and  confidence  between  the  UK  and  US  crews  was  starting  to  be  eroded  during  the  trial. 

Exercise  management:  Trial  SyCLONE  provided  a  first  opportunity  to  test  an  established,  colocated,  WF 
model  in  a  distributed  environment;  this  model  has  been  developed  within  the  research  programme  and  used 
successfully  in  previous  colocated  trials. 

SyCLONE  has  shown  that  the  WF  model  used  in  previous,  successful,  networked  trials  cannot  be  applied  if 
participants  are  distributed.  Careful  consideration  needs  to  be  given  on  whether  or  not  an  alternative  exercise 
management  model  could  be  adopted  for  a  real  distributed  collective  training  system.  One  option  may  be  to 
have  WF  ‘agents’,  or  WF  liaison  officers  (WFLOs),  embedded  with  the  aircrew.  Each  WFLO  would  liaise  with 
the  central  WF  team  on  the  overall  gameplan  and  general  thrust  of  the  mission.  WFLOs  would  then  monitor 
aircrew  during  the  planning  phase  and  guide  planning  direction,  where  and  if  appropriate,  within  the  WF 
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Commander’s  directive.  WFLOs  could  also  pass  on  relevant  information  on  aircrew  performance  to  the  central 
WF  team. 

To  summarise,  running  a  distributed  training  exercise  is  more  difficult  than  running  a  colocated  one.  It  is  not 
possible  to  simply  apply  the  model  which  has  been  used  successfully  in  colocated  trials  to  the  distributed 
situation.  Specifically,  an  appropriate  exercise  management  model  needs  to  be  developed.  A  central  exercise 
management  team  cannot  have  direct  oversight  of  multiple  dispersed  sites.  In  order  to  maintain  co-ordination 
there  may  be  a  requirement  for  WFLOs  at  each  site  but  this  needs  to  be  tried  and  tested.  As  yet  there  is  no 
evidence  that  the  WFLO  model  would  work  in  practice.  However,  it  is  planned  to  use  WFLOs  in  the  NATO 
SAS34  MTDS  exercise,  First  WAVE,  scheduled  to  take  place  in  September  2004.  This  exercise  should  be 
monitored  closely  to  learn  as  much  as  possible  about  the  feasibility  of  using  WFLOs  in  large  scale  distributed 
training  exercises. 

SyCLONE  -  overview  of  findings 

There  is  a  limit  to  the  conclusions  that  can  be  safely  drawn  from  this  trial.  SyCLONE  was,  after  all,  a  single 
trial  based  on  a  small  sample  of  aircrew.  However,  SyCLONE  gave  valuable  insight  into  many  of  the  issues 
associated  with  running  a  distributed  training  exercise.  It  is  a  much  more  difficult  prospect  than  any 
encountered  previously  in  the  research  and  simply  applying  the  RAPTORS  model,  known  to  work  successfully 
in  colocated  trials,  did  not  work  for  the  distributed  situation. 

In  simple  terms  the  trial  can  be  said  to  have  been  a  success  in  that  the  available  technology  supported  a 
distributed  exercise,  in  which  aircrew  planned,  briefed,  flew  and  debriefed  a  complex  mission.  However  the  real 
issue  is  to  understand  how  effective  such  an  exercise  could  be  in  providing  high  value  mission  training.  The 
difficulties  aircrew  experienced  might  be  attributable  to  distribution,  to  artefacts  of  the  technology  chosen  for 
the  trial  or  simply  to  lack  of  experience  in  using  such  technology.  Some  of  the  observed  adverse  effects  might  be 
overcome  by  the  application  of  simple  expedients.  The  trial  has  however  clearly  demonstrated  that  it  would  be  a 
mistake  for  MTDS  protagonists  to  assume  that  it  is  simply  a  case  of  ‘provide  linking  technology  and  all  will  be 
well.’ 

If  there  is  a  genuine  causal  link  between  poor  mission  performance  and  distribution  then  it  is  important  to 
understand  why  this  should  be  the  case.  One  possibility  is  that  the  planning  constraints  already  described  had 
an  adverse  effect  on  the  mission  itself.  It  can  be  readily  understood  that  these  could  influence  the  tactical  nature 
of  the  plan  with  undoubted  consequences  for  mission  outcome.  In  order  to  test  this  hypothesis  it  would  be 
necessary  to  eliminate  the  known  planning  constraints  and  observe  the  effect  on  overall  performance. 

Conclusions 

The  MoD  sponsored  research,  coupled  with  military  opinion,  strongly  suggests  that  there  is  much  benefit  in 
distributed  mission  training  (i.e.  MTDS).  It  has  the  potential  to  provide  guaranteed  COMAO  training.  The 
question  is,  what  is  an  optimised  MTDS  configuration;  all  assets  dispersed,  all  colocated  or  a  compromise 
between  the  two?  MTDS  does  not  necessarily  mean  that  the  training  systems  have  to  be  de-centralised.  It  is  equally 
possible  to  conduct  training  with  synthetic  training  devices  all  at  the  same  site,  linked  together  via  a  LAN.  The  devices 
woidd  still  all  be  distributed  albeit  on  the  same  network.  However,  if  the  aspiration  is  to  undertaken  coalition  mission 
training  some  elements  of  MTDS  will  have  to  be  geographically  dispersed.  This  would  add  additional  levels  of 
complexity  due  to  interoperability  issues  that  may  severely  impact  upon  the  quality  of  training  that  could  be  undertaken. 
Research  findings  on  the  impact  of  distribution  are  not  straightforward  but  a  fully  distributed  MTDS  solution 
would  sacrifice  some  of  the  intrinsic  advantages  of  colocation. 
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Irrespective  of  the  final  MTDS  solution  (colocated  and/or  comprising  geographically  dispersed  elements)  the 
requirement  is  first  fully  scope  out  the  problem  space  in  terms  of  the  cost  versus  training  requirement.  This 
necessitates  a  thorough  appreciation  of  both  the  technical  and  training  interoperability  issues  associated  with 
MTDS.  It  is  imperative  that  there  is  inter-system  and  inter-team  component  compatibility  and  interoperability. 
Careful  thought  must  be  given  to  the  most  appropriate  network  architecture.  In  addition  there  are  significant 
security  implications  for  a  military  exercise  being  conducted  in  a  dispersed  STE.  The  optimal  technical  solution 
for  a  distributed  mission  training  capability  which  fully  supports  the  operational  needs  of  the  front-line,  yet  is 
cost-effective,  and  meets  all  the  necessary  doctrinal  and  security  requirements  has  yet  to  be  defined. 
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